The present invention relates to a post-processing device for eliminating a blocking artifact and a method thereof, and more particularly, to a post-processing device for minimizing a blocking artifact where a discontinuity occurs among adjacent blocks in a device for restoring image data compressed by a block transform operation, and a method thereof.
There are various standard methods for encoding video (and audio) signals into a digital signal for transmission or storage, and decoding the transmitted or stored result for reproduction. These methods, however, produce very large amounts of encoded digital video data. Therefore, in order to decrease the total amount of data, redundancy contained in the digital image signal is eliminated by coding techniques such as transform coding, differential pulse code modulation, quantization, and variable-length coding.
FIG. 1 is a block diagram schematically showing a general coder of image data. The device roughly comprises means 11 and 12 for transforming an input image signal into a frequency domain signal by block units and quantizing a transform coefficient, means 13 and 14 for variable-length-coding the quantized data, and means 15, 16, 17, 18, 19, A1, A2, SW1 and SW2 for inverse quantizing and performing an inverse-transform operation on the quantized data so as to perform motion compensation, to thereby encode image data in an intramode or in an intermode.
In addition, FIG. 2 is a block diagram schematically showing a general decoder and a post-processing device of image data, for decoding image data which has been coded by the coder shown in FIG. 1 and reproducing the decoded result. The operation of the circuit shown in FIG. 2 is as follows.
Coded image data QF(u,v) is decoded by a variable-length decoder 21 for output to an inverse quantizer 22. Here, the magnitude of the output transform coefficient is controlled by a quantization step size Qss provided by the coder (FIG. 1). Then, an N.times.N inverse transform operator 23 performs a transform operation on a transform coefficient of a frequency domain provided by inverse quantizer 22 to output image data of a space domain.
The data compression with respect to each block in N.times.N inverse transform operator 23 shown in FIG. 1, employs a discrete cosine transform (DCT), Walsh-Hadamard transform (WHT), discrete Fourier transform (DFT) or discrete sine transform (DST) method. For example, if data compression is performed by employing the DCT method, a two-dimensional forward DCT function (equation 1) and an inverse DCT function (equation 2) with respect to pixel P.sub.(yx) can be defined as follows. ##EQU1## Here, u, v, y and x are integers from 0 to 7, and C.sub.u and C.sub.v are both equal to 1/.sqroot.2 when u and v are both zero and are otherwise equal to one.
A motion vector MV from the coder (FIG. 1) is provided to a motion compensator 24 of a decoder. Motion compensator 24 reads an N.times.N block corresponding to the motion vector from the previous frame data stored in a frame memory 25, performs motion compensation and provides the result to an adder A3. The adder A3 sums the inverse DCT data and the N.times.N block data and outputs the result to a post-processor 26 for restoring the original image from a received distorted image. That is, if the received signal is image data compressed by block units, blocking artifacts are generated. A blocking artifact is a discontinuity which occurs among adjacent blocks and is a form of distortion.
Therefore, in a conventional device, to reduce the generation of blocking artifacts, a low-pass filtering operation is performed spatially for boundary pixels. However, post-processing in a transform domain rather than a space domain simplifies the decoder structure, and it is also effective.
Accordingly, as disclosed by Chung-nan Tien and Hseuh-ming Hang in "Transform-domain Postprocessing of DCT-coded Images" ('93 SPIE Proceedings, Vol. 2094, pp. 1627-1638), post-processing is performed using a compensated DCT coefficient, which will be explained with reference to FIG. 3 to FIG. 7.
FIG. 3 is a flowchart illustrating the post-processing method.
Referring to FIG. 3, received quantized blocks are inversely quantized (step 1), and a processed segment consisting of the current block and its eight neighboring blocks, as shown in FIG. 4, is classified is into smooth or edge segments (step 2). Since a blocking artifact is especially annoying in a smooth segment, post-processing (steps S3 to S5) is performed if a processed segment is smooth, and the processed segment is inversely transform-operated so as to output reconstructed image blocks if the processed segment is an edge segment (step S6).
First, the post-processing (steps S3 to S5) will be explained in more detail.
If the classified segment is a smooth segment, a DC value is adjusted (step S3). Here, since the DC value indicates an average luminance of the image block, an overall image block luminance level can be changed by adjusting the DC value. In other words, a weighted averaging method for adjusting the DC value of the current block based on information of the adjacent blocks is employed to reduce discontinuity (blocking) among adjacent blocks. The DC value of a block.sub.i,j (DC.sub.i,j) can be replaced with an adjusted DC value DC.sub.adj(i,j), for a smooth transition between adjacent blocks. Here, the adjusted DC value can be calculated as follows. ##EQU2## Here, the weighted mask (WM), shown in the following expression (4), is selected by experience. ##EQU3##
Though the average image intensity can be changed by adjusting the DC value, as described above, discontinuity may still remain, even when the discontinuity of the DC value of the adjacent blocks is lowered by the DC value adjustment, and a blocking artifact cannot be completely eliminated. Therefore, two additional transform coefficients need to be adjusted, inversely transform-operated and output. That is, the vertical and horizontal AC coefficients VAC and HAC, i.e., DCT coefficients (0,1) and (1,0), are adjusted (steps S4 and S5). Since the HAC and VAC operations are very similar, only the VAC operation will be described.
Using the forward DCT function of the above equation (1), the vertical AC coefficient for S.sub.(v,u) (where v=0 and u=1) is calculated. ##EQU4## Here, S.sub.(0,1) is a vertical AC coefficient (VAC). The amount of pixel values in a space region, which is affected by the vertical AC coefficient is calculated by an inverse DCT operation of the above equation (2). ##EQU5##
As can be seen in the above expression (6), a vertical AC coefficient is related only to the magnitude variation in the horizontal direction modulated by a cosine function. Accordingly, the VAC value reduces a blocking artifact across a vertical boundary and reestablishes continuity in the horizontal direction.
As shown in FIGS. 4 and 5, block.sub.i,j-1, block.sub.i,j and block.sub.i,j+1 are horizontally adjacent blocks and thus require VAC adjustment. The VAC adjustment is employed to remove a blocking artifact that remains after DC adjustment. Therefore, it is necessary to measure a gap between two adjacent blocks at their boundaries. Under the assumption that there is no acute variation in a block, the above expression (6) is employed to estimate a gap through the following processes.
First, intensities P.sub.i,j (left) and P.sub.ij (right) correspond to the left and right boundaries of block.sub.i,j and can be expressed as follows. ##EQU6##
Then, the right boundaries of block.sub.i,j-1 and the left boundaries of block.sub.i,j+1 can be expressed as follows. EQU P.sub.i,j-1 (right)=0.17654VAC.sub.i,j-1 ( 9)
and EQU P.sub.i,j+1 (left)=0.17654VAC.sub.i,j+1 ( 10)
The gaps between adjacent block boundaries can be calculated as follows. EQU gap.sub.i,j (left)=P.sub.i,j-1 (right)-P.sub.i,j (left) (11) EQU gap.sub.i,j (right)=P.sub.i,j (right)-P.sub.i,j+1 (left) (12)
FIG. 6 shows an intensity and a gap between boundary intensities.
An average gap (gap.sub.ave(i,j)) between the current block boundary and adjacent blocks can be expressed as follows. ##EQU7##
If the current block and the adjacent blocks have pixel values which are continuous in the horizontal direction, the value of gap.sub.ave is zero. A large gap.sub.ave value indicates a large discontinuity. Therefore, the discontinuity can be lowered by reducing the value of gap.sub.ave. The gap adjustment is made by halving the average gap; that is, gap.sub.adj =gap.sub.ave /2. To reduce the gap of a boundary of block.sub.i,j, ##EQU8##
An intensity of a block boundary after VAC adjustment is shown in FIG. 7. Here, HAC adjustment would be performed to reduce the vertical discontinuity in a similar method.
Since the above-described post-processing method employs a weighted average value for the DC value of peripheral blocks during adjusting DC value, and performs horizontal and vertical adjustments independently upon calculating horizontal and vertical AC adjustment values HAC and VAC, this method is not very effective for eliminating a blocking artifact.